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Abstract 

The influence of a time-periodic forcing on stochastic processes can 
essentially be emphasized in the large time behaviour of their paths. The 
statistics of transition in a simple Markov chain model permits to quantify 
this influence. In particular the flrst Floquet multiplier of the associated 
generating function can be explicitly computed and related to the equi- 
librium probability measure of an associated process in higher dimension. 
An application to the stochastic resonance is presented. 
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Introduction 

The description of natural phenomenon sometimes requires to introduce stochas- 
tic models with periodic forcing. The simplest model used to interpret for in- 
stance the abrupt changes between cold and warm ages in paleoclimatic data is 
a one-dimensional diffusion process with time-periodic drift [3]. This periodic 
forcing is directly related to the variation of the solar constant (Milankovitch 
cycles). In the neuroscience framework, such periodic forced model is also of 
prime importance: the firing of a single neuron stimulated by a periodic in- 
put signal can be represented by the first passage time of a periodically driven 
Ornstein-Uhlenbeck process [T2] or other extended models [5]. Moreover let us 
note that seasonal autoregressive moving average models have been introduced 
in order to analyse and forecast statistical times series with periodic forcing. 
Recently the time dependence of the volatility in financial time series leaded to 
emphasize periodic autoregressive conditional heteroscedastic models. Whereas 
several statistical models permit to deal with time series, the influence of pe- 
riodic forcing on time-continuous stochastic processes concerns only few math- 
ematical studies. Let us note a nice reference in the physics literature dealing 
with this research subject [5]. 
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Therefore we propose to study a simple Markov chain model evolving in 
a time-periodic environment (aheady introduced in the stochastic resonance 
context [7| and [5]) and in particular to focus our attention to its large time 
asymptotic behaviour. Since the dynamics of the Markov chain is not time- 
homogeneous, the classical convergence towards the invariant measure and the 
related convergence rate cannot be used. One essential tool is to increase the 
dimension of the state space in order to construct an appropriate homogeneous 
Markov chain and to apply classical ergodic results. 

Description of the model. Let us consider a time-continuous Markov chain 
evolving in the state space S = { — 1,1} whose transition rates correspond to 
(fi^ respectively (p^, the exit rate of the state —1 resp. -f 1. We assume that 
ipj. > and we perturb this initial process by a periodic forcing of period T; it 
means that the transition rates are increased using two additional non negative 
periodic functions (p^. The obtained Markov chain is denoted by (Xt)t>o and 
its infinitesimal generator is given by 



(0.1) 



where (p±{t) = (p^. + 'p±{t) are T-periodic functions. In order to describe pre- 
cisely the paths of the chain (Xt), we define transitions statistics: Aft corre- 
sponds to the number of switching from state —1 to -fl up to time t. The 
law of the random process Aft at a fixed time t is characterized by its moment 
generating function: 

vl/(77,t) :=E[7?^'], 77 >1. (0.2) 

Main result. Let us first note that, in the higher dimensional space [0, T] xS, 
we can define a Markov process (t mod T.,Xt)t>o which is time-homogeneous 
and admits a unique invariant measure fi = (/i_ (t), /i-|-(t))o<t<T- The main 
result can then be stated. The periodic forcing implies the use of Floquet's 
theory to obtain a precise description of the moment generating function: there 
exist two time-periodic functions Pi{ri, t) with i = 1,2 such that 

*(77,t) -Pi(7],t)e^i*+P2(??,t)e^^*. (0.3) 

where pi is a positive function, Ai > 1 > A2 and 

Ai = ^ (^_(s)^_(s)ds, X2 = ^J^ ip-{s)+ip+{s) ds- Xi. (0.4) 

This result implies in particular that, if the observed time-interval [0, t] is in- 
creased by adding a period, the associated function 5'(?/,t) is multiplied by a 
parameter which becomes close to 

as t becomes large. 

Application. The explicit expression (|0.4p of the first Floquet exponent Ai 
permits to deal with particular optimization problems appearing in the stochas- 
tic resonance framework (see, for instance, [S]). Let us consider a family of 
periodic forcing having all the same period T and being parametrized by a vari- 
able e, then it is possible to choose in this family the perturbation which has 
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the most influence on the stochastic process, just by minimizing the following 
quality measure: 



Ja 

In Section [3] we shall compare this quality measure (already introduced in |13| ) 
to other measures usually used in the physics literature [7]. 

1 Periodic stationary measure for Markov chains 

Before focusing our attention to the paths behaviour of the Markov chain, we 
describe, in this preliminary section, the fixed time distribution of the random 
process and, in particular, analyse the existence of a so-called periodic stationary 
probability measure - PSPM (we shall precise this terminology in the following). 
The distribution of the Markov chain {Xt)t>o starting from the initial distribu- 
tion vq and evolving in the state space {—1,1} is characterized by 



This probability measure i> = (i^_, z^+)* (the symbol * stands for the transpose) 
constitutes a solution to the following ode: 



where the generator Qt is defined in (|0.ip . 

Floquet's theory dealing with linear differential equation with periodic coeffi- 
cients can thus be applied. In particular we shall prove that iy{t) converges 
exponentially fast towards a periodic solution of (|l.ip . the convergence rate 
being related to the Floquet multipliers (see Section 2.4 in [2]). 

Definition 1.1. Any T -periodic solution i'{t) = (v-lt), i^+(t))* of p.ip is called 
a periodic stationary probability measure - PSPM iif v± (t) > and (t) -\- 
u+{t) = 1 both for allt>0. 

The following statement points out the long time asyniptotics of the Markov 
chain. 

Proposition 1.2. In the large time limit, the probability distribution v con- 
verges towards the unique PSPM fj, defined by fj,(t) = (^_(t), 1 — H-(t)) and 




z.±(0 =P.o(Xt -±1). 



di^it) 



Qti^{t) and v{0) = 



(1.1) 



dt 




(1.2) 



where 




(1.3) 



More precisely: 



lim - \ogMt) - ^l{t)\\ < A2 

r— >-oo t 



(1.4) 



where X2 stands for the second Floquet exponent: 




(1.5) 
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Remark 1.3. It is possible to transform (Xt) into a time-homogeneous Markov 
process just by increasing the space dimension. By this procedure (^(i))o<t<T 
becomes the invariant probability measure of (t mod T,Xt)t>o- 

Proof. 1. First we study the existence of a unique PSPM. Let be a prob- 
ability measure thus p-{t) + /-«+(<) = 1. If /-t satisfies p.ip then we obtain, by 
substitution, the differential equation: 

^M-W = -<^-(tV-(t) - A^-(t)). 

This equation can be solved using the variation of the constant. The procedure 
yields to (|I.2p . The periodicity of the solution requires p-{T) = H-{0) and 
leads to (jO]). 

2. The system (jl.ip admits two Floquet multipliers pi and p2- Since there 
exists a periodic solution, one of the multipliers (let's say pi) is equal to 1 and 
we can compute the other one using the relation between the product pip2 and 
the trace of Qf. 

piP2 = exp ^ j iv{Qt) dt^ . 

The explicit expression of the trace leads to (|1.5p . Let us just note that we can 
link to both Floquet multipliers pi and pi the so-called Floquet exponents Ai 
and A2 defined (not uniquely) by 

pi = e^i^ and p2 = e^"^. 

3. Since the Floquet multipliers are different, each multiplier is associated with 
a particular solution of (|l.ip . pi = 1 (i.e. Ai = 0) corresponds to the PSPM 
since p{t + T) = pip{t) for all t G M+. For the Floquet exponent A2, we consider 
p{t) the solution of (jl.ip with initial condition p(0)* — (—1, 1). Combining both 
equations of (|l.ip . we obtain 

p_(t)+p+(t)=0 

p_(t)-p+(t) = -2cxp-/o((^_+(^+)(s)ds. ^ ■ ' 

We deduce 

p(i)* = ^- cxp - y" {f^ + Lp+){s)ds, cxp- J {if_+(p+){s)ds 
and we can easily check that p{t + T) = p{t)e'^'^^ . 

The solution of (jl.ip with any initial condition is therefore a linear combination 
of p and /i, the solutions corresponding to the Floquet multipliers. Writing i/(0) 
in the basis (/i(0),p(0)) yields v{t) = ap{t) -\- /3p{t), with a = i^+(0) + i^-(O) 
(equal to 1 in the particular probability measure case) and 

t.+ (0)-z.-(0) /(^+)-/(y_) 
^ 2 ^ 2/((/p_+<^+) ■ 

Then, if the initial condition is a probability, we obtain (jl.Sp since 

Mt)^pm - \\f3p{m^V2me-^oi'p-+<p^K^)'is_ 

□ 
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2 Statistics of transitions 



In the previous section, the study of the process points out how fast its distri- 
bution converges towards a periodic stationary distribution (in the sense that 
Xf and Xt+T are identically distributed). The aim now is to improve this re- 
sult, which is just related to the position of the chain at some fixed time t, by 
analysing the paths behaviour in the large time scale, especially the statistics 
of the transitions between the two states namely —1 and -1-1. That's why we 
introduce the moment generating function 

V'(7?,0=IE[,Al (2.1) 

associated with Aft, the number of transitions from state —1 to state 1 up to 
time t. We can decompose this generating function into two parts: 

i}{Tj,t) = ip_{Tj,t) +^+{7j,t) where i{j±{r],t) = E r]^'l[Xt=±i} 

Then the vector 'i>{i],t) = {ijj^{'i'i,t),ip+{'ri,t))* satisfies the ode: 

'"^^''^QMnv^t), with Q(,,t).r:r-w ^^^V (2.2) 



Since (|2.2p is a differential equation with periodic coefficients, Floquet's theory 
can be applied and, in this way, the Floquet multipliers describe the large time 
behaviour of any solution to the equation (j2.2p . in particular the generating 
function. That's why we shall compute explicitly these multipliers. In Section 
[ijthe system of ode considered has been reduced to a one-dimensional equation, 
here it is not the case so that we need to introduce an other procedure: a 
time-discretization approach. 

2.1 The discrete-time counterpart of the chain (Xt)t^^_^ 

We consider a non- homogeneous Markov chain in discrete time {Z^ , n G N) 
defined on the space state S = {—1,1}. The associated transition matrix is 
given by 

ttN _ f 1 ^ ''^n.N ''^n.N 
" ^ I TT- 1 - 7r+ 

where (7r^jv)">o are periodic sequences of period N . Let us introduce different 
quantities related to (tt^jv)- 



N-l 

:= 1 - {'^Xn + Vw) and := l{fe>Ar} + l{o<fc<7V-i} H ' (2-3) 

j=fe 

For notational simplicity, the index shall voluntarily be removed when there is 
no ambiguity. Let us now consider usual properties of the discrete-time Markov 
chain: existence of periodic stationary probability measure, uniqueness and er- 
godic properties. 
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Proposition 2.1. The Markov chain {Z^ , n mod N)n>Q admits a unique 
stationary probability measure fi^ given by 

AN AN 

= 5^m"(-1,0) + E <N^ (2.4) 

" fe=0 " 

and 

^JV-l + .AT 



(-1,0) = ^ _ ■ (2.5) 

Remark 2.2. Since the PSPM is a probability, we compute l,n) and 

deduce ^^(1, n). 

Proof. Let us define the probability tliat tlie cliain Z„ is in the state —1 at 
time n. The cliain is initialized through = ¥{Zq = —1). Then applying the 
matrix Hq, the distribution of Zi satisfies i^i = aoi'o + ''I'o^- The same argument 
yields the distribution of Z2: 

V2 — aiL'i + = aiaQUQ + aiTT^ + -k^ . 

By induction, we obtain the general formula: 

(n — 1 \ n — 1 / n — 1 

[| a, + XI Mfc n 
]=0 J fe=o \ i=fc+i 

which can be rewritten by use of the quantities AJ? as follows: 

^N y^A' 
^" fc=0 ^" 

Since the Markov chain [Z^ , n mod N)n>Q is positive and recurrent, there 
exists a unique stationary probability measure fi. This measure is obtained by 
setting 1, 7T.) = Vn and solving I'jv = ^^o- This leads immediately to the value 
of i/Q and in the sequel i^n for any 1 < n < using (j2.6p . □ 

Theorem 2.3. Let be the number of transitions from state —1 io state +1 
performed by the Markov chain {Z^)n>Q up to time n (included). We denote 
by ^^(r^, n) its moment generating function: 

v['^(77,n):=E[ry^"]. (2.7) 
The following asymptotic behaviour holds: 

hm ilog*^(r;,n) = 1^ ^'."^^^(-l, fc) = [A/"^], (2.8) 

A:=0 

where is the stationary distribution of the Markov chain {Z^ , n mod N)n£m 
defined in Promsition \2. 1\ 
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Proof. Step. 1 Let us prove that for any j > 



V[-^'^]=E^^.^M'^(-l'fc)- (2-9) 

fc=0 

The second equahty in (|2.8p is then immediate. We obtain (|2.9p by induction: 
for j = I, we easily have 

E^«[AAf ] = P,,«(Zo = -1, = 1) = A*^(-l,0)7ro^^. 



We assume that (|2.9p is satisfied for some > 1, then 

This leads to (|2.9p with j replaced by j + 1. Formula (|2.9p is therefore satisfied 
for all j especially for j ~ N. 

Step 2. The ergodic theorem is the essential tool for studying the long time 
behaviour. First of all we construct a new {—1, l}^-valued Markov chain (y„)„>i 
defined by 

Yji = {Zn~i, Z„). 

The associated transition matrix depends on tt^ and n; moreover it is A'^- 
periodic. To deal with an homogeneous chain, it suffices to consider (Yn, n mod N). 
Suppose that the law of Z„ is the invariant periodic measure describe in Propo- 
sition [^TT] then Yn is also in the invariant regime: for all {a,b) G {—1,1}^, we 
have 

P(r„ = (a, 6)) = P(Z„_i = a, Zn = 6) = fi{a,n- 1)P(Z„ = 6|Z„_i = a). 

In the previous expression, both terms constituting the right hand side are A'^- 
periodic. Hence P(y„ = (a, 6)) is periodic. Let us therefore define the following 
measure ii^ : 

fi^i-l, -l,n) = - 7r,;_i) 

+1, n) = n - l)7r„_i 

M^(+l,-l,7i) = /x(+l,n-lV+_i 

= 1)(1 - 7r+_i). 

Let us just observe that jjX is a positive invariant measure for the homogeneous 
Markov chain (y„, n mod N). It suffices to normalize the measure in order 
to obtain an invariant probability: -^M^. Let us now consider the moment 
generating function ^(r],n) associated with the number of transitions Afn- By 
construction, Afn is also the number of time the non-homogeneous Markov chain 
visits the state (—1, 1) or finally the number of times the homogeneous Markov 
chain n mod N) visits the set 



A 



{(-1,1,1), (-1,1,2), ...,(-l,l,iV)}. 



Since the homogeneous chain is recurrent positive and aperiodic, the ergodic 
theorem implies 

N 1 1 
„1™ V = N^'^^^'^ =N^ ""^^'^-^^ 

fe=0 
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More precisely the law of the iterated logarithm implies the existence of a con- 
stant 7 > such that 



lim sup ■ 



Nn-^,tJ^{A) <7 a.s. (2.10) 



N' 



\/n log log n 

Let us then define the event the set of all paths satisfying 

1 



< 



1- 



yjn log log n 

Hence, for < 1, we obtain the associated decomposition 

^(7,,n)=]E[r;^"loJ+E[r;^"lnc]. 

Using Lebesgue's dominated convergence theorem and (j2.10p . the second term 
converges to when n tends to infinity. Moreover 

P(aO exp { (^y^{A) - log log n) log(?/)} < Eiry^-laJ 

and by symmetry 

E[77^"lj^J < P(r!„) exp |(^^^(^) + 701 log log n) log(r,)}. 

Since P(ri,i) converges to 1 as n — > oo, the combination of both previous in- 
equalities leads to 

lim -log*^(7?,n) =log(7])4^^(yl). 
In order to conclude the proof, it suffices to compute the explicit expression of 

In a similar way to Section [U the asymptotic behaviour of the moment 
generating function is related to the eigenvalues of a suitable matrix. Indeed, if 
we decompose the generating function as follows 

*^(?7,n) = vl;^(77,n) + vl/^(7y,n) with ^^(ry, n) = E[r;^"l{;j«=±i}], (2.11) 

then the vector 4'(n) :— {^^{ri,n),'i/^{ri,n))* satisfies the recurrence relation: 

*(n + l)=M„^*(n) where M„^ = f \" !"'^ ). (2.12) 

Let us define the product of matrices 

M"" = M^_^M^_^ . . . M^. (2.13) 

We thus obtain ^{N) ~ A^^\I'(0). We observe in the following statement the 
link between the asymptotic behaviour of the moment generating function and 
the eigenvalues of the monodromy matrix. 
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Corollary 2.4. Let rj > 1. The eigenvalues of the monodromy matrix M 
satisfy: \^ > 1 > and 

N-l 

log(Af ) = log(r;) J] ^,7^/i^(-l, k). (2.14) 

fe=0 

Proof. The matrix is directly related to the fundamental solution of equa- 
tion (|2.12p . In fact its coefficients correspond to 

nv'^-l^z-^a^Z^ ^b] With (a,fe)e{-l,ir. 

These coefficients are all positive: the matrix thus admits two distinct 

real eigenvalues, denoted by \i and (Af corresponds to the largest one), 
and is diagonalizable. We denote by Ui, i = 1,2 the associated eigenvectors. 
Hence ^(0) can be expressed in this basis: there exists ri and r2 such that 
*(0) = riUi + r2U2. We immediately deduce 

*(fciV) ri(Af )'^-Ui +r2(A^)'^U2, Vfc > 0. (2.15) 

Let us prove that A^ > 1 > A^^. If we compute the determinant of , we 
obtain: 

det(A'/,f ) = 1 - 7r+ - TT,^ + (1 - 7?)7r,+7r,7 = q„ + (1 - 7?)7r+7r,7. 
We recall that a„ is defined in (gSl). Since log(Af A^) = log(det(7W^)), 

log(Af A^) = log(dct(A/f )) = J2 ("fc + (1 - ^^H^k) ■ (2-16) 

fc=0 k=Q 

Due to the assumption 77 > 1, (j2.16p leads to log(A;j^A^) < 0. Suppose that 
log(Af ) < (consequently log(A^) < 0), then according to ((2T^ . (*(fc7V))fe>o 
is a bounded sequence. Of course, this is a nonsense since "^(kN) is the gener- 
ating function (associated with ry > 1) of a growing process which a.s. tends 
to infinity. Consequently log(A^) > 0, log(A^) < and vi 7^ 0. Finally we 
deduce the large time asymptotic behaviour: 

hm — log(*(fciV), 1) = (2.17) 

where 1 = (1, 1)*. Using ((2ll|l and Theorem [231 we obtain ((2T4|) . □ 

Remark 2.5. It is also possible to compute the second eigenvalue of the mon- 
odromy matrix M.^ . Indeed, by (j2.16p . we get: 

N-l N~l 

log(A^) = J2 log ("fe + (1 - ^H^k) - log('?) E 

fc=0 k=0 
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2.2 On the convergence from discrete to continuous time 

In the previous section, the asymptotic behaviour of a periodic discrete-time 
Markov chain was emphasized. The study of such Markov chain was a first step 
in the analysis of continuous-time Markov chain. We shall now describe how all 
previous results can have a continuous counterpart. 

In this section, we especially prove that the Markov chain , the expression 
defined by (|2.3p . the probability measures fi^{—l, n) defined by (|2.4p and (|2.5p 
and finally the moment generating function ip^{r],n) converge as N becomes 
large. 

Let us assume that the transition probabilities of the Markov chain are 
small with respect to 

^7n - I? V- (^t) and ^+ (i-r] for all j e N (2.18) 



where and ip-i- are piecewise continuous functions defined in (jO.ip . 

Lemma 2.6. Lett > 0. Under the assumption (|2.18p . the following convergence 
holds uniformly w.r.t. the variable t 



N 



(-/f(^-+^+)(s)ds) */ 0<t<T 

otherwise. 



limylj^^j =<! "^'''' ^ ^''^ ' ''"""^"V \_ - (2.19) 



We recall that is defined in (|2.3p . 

Proof. For notational simplicity, we set k = [yiVj. By definition A^ = 1 for 
k> N. Using (|TT5| . we obtain for < fc < TV - 1, 

iog<=x:'iog(i-§(^-+^+) (f 

j=k 



Using the Taylor expansion, we get 



N ^ \ " 



N ^ 



To conclude it suffices to use the uniform convergence of the Riemann series the- 
orem on one side and to prove that the error term converges uniformly towards 
0. The details are left to the reader. □ 

Since we have a link between the transition probabilities of the discrete-time 
Markov chain (through A^ ) and the transition probabilities of the continuous 
one we shall compare the processes themselves. First we investigate the 
comparison of the stationary measures and secondly we point out the conver- 
gence in law of the processes. 

Let us define the time-continuous process (Z^(t))t>o associated with the dis- 

'k 



Crete Markov chain (^^, fc mod N)k^fi as follows 



Z^(t) , for <t<{k + 1)^, fc e N. (2.20) 

Let us note that {Z^ {t))t>o is a piecewise constant and cadlag process. 
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Proposition 2.7. The stationary probability measure (t) associated with the 
process {Z^{t),t mod T)t>o converges to the stationary distribution ^{t) of the 
Markov chain {Xt,t mod r)t>o. This convergence holds uniformly w.r.t. the 
time variable t. 

Remark 2.8. We shall use the following result : if Bf^tj^. converges uniformly 

to B(t) on [0,T] and if (j) is piecewise continuous on [0,r], then the following 
convergence holds uniformly 

N-l , i-T 



Proof of Proposition \2.7[ We first focus our attention to the convergence of 
/i^(-l,0) given by (^3)) . Tlie assumption fUE\i leads to 

N-l N-l . . 

E^M^i^lEv'+ff )<i (2.21) 

k=0 k=Q ^ ^ 

wliicli converges uniformly (Lemma I2.6P towards 

^+{s)e- ST(v~+v+)Md^ds = (2.22) 



Let us just recall that /(•) has been defined in Using (^3]) . (EH]), ^^TF^ 

and the identity 1 — = li^P- + </?+), leads to 

lim //^(-1,0) = /x_(0). (2.23) 

AT-i-oo 

Using similar arguments, we prove the convergence of /i^(— l,fc) defined by 
(|2.4p . Let t and s two real numbers such that t > s > 0. We define k and j by 
k = [^A^J and j = [^A^J. We observe the following uniform convergences: 



J™,E<^^ = X (2.24) 



JToo If ^ '^''P / + ^+)(-^)^5) ■ (2-25) 
By ^T^, (p:^ and (P?^ apphed to ([131), the uniform hmit holds: 

i-t 

lim /^^(-l,fc) = e-^o('/'-+v+)W'i^^_(0) + / (p+(s)e-/=('^^+'^+)(")''"ds. 

(2.26) 

The right hand term corresponds to the expression of /-t-(i), see (|1.2p . □ 

Proposition 2.9. T/ie process {Z^{t), t mod T)t>o, defined by (j2.20p . con- 
verges in distribution towards the Markov chain {Xt, t mod T)t>o. 
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Proof. The proof is divided into three parts. First we prove the convergence of 
the conditional distribution of successive jumps. In the second part, we prove 
the convergence of the process on any bounded intervaL Finally, we point out 
that converges in distribution to X. 

Step 1. Convergence of successive jumps. We set Z^{Q) = Zq = —1. We 
iZ^{t))t>o as 



set ^ To = and we define the successive transition times for the chain 



= inf{t > r„^_i : z^it) ^ z^(r„^_i)}, 

and for the chain {Xt)t>a the associated transition times are denoted by T„. 
For s > (resp. t > 0), we set i = [j^N\ (resp. k = L^^^^J)- We study the 
transition times from —1 to 1: by (j2.18p we get 

'rpN _ 

^■2n - 



Ri,k IP(72„+i - T^n > ^"^211 — i) — IP y^T^f^ + l — • • ■ — ^Tj" +fc 
i+k-1 i+k-1 , rp / rp 

= n(i-^-.)= n (i-§^-(f 
j=i j=t ^ ^ 

By similar arguments as those developed in Lemma |2.6[ we obtain 

^lirn^ R,,k = exp ^ if. {u)du^ = ¥{T2n+l - T2n > t\T2n = s). (2.27) 

For transitions from 1 to —1, that is typically — 2^2ri-i given T^^_i, we 
obtain the same result, just replacing ip- by tp^. 

Step 2. Let us prove now the convergence of the Markov chain on any compact 
set. Wet set the time interval [0,1], it is straightforward to generalize to any 
compact set. Let us define J\f^{t) the counting process of all transitions of the 
chain (Z^(.s))s>o on the interval [0,t]: 

AA^(i)=^l{Tf<t}, te[0,l], (2.28) 
fc>i 

and Af the counting process associated with the chain {Xt)t>o. We prove that 
the distribution sequence (Piv)Ar>i of the processes A/"^ is tight (see for instance 
Theorem 13.3 p. 141 in [T]). The theorem requires two conditions. The first one 
is 

lim FQXs - Xil > e) = 0, for any e > 0. (2.29) 

(5-i.l 

Let us define dg^P (Vt E [I - S,l], Xt = Xi). Then 

d5 = ^P(Vte [l-(5,l],Xt = zl) = ^P(Xi_i =^l)exp (- f 
i=± i=± ^ "'1- 

> exp I — (5 max I sup (^-|_(t), sup (y9_(t))). 



ipi{s)ds 

s 



Thus we obtain 



lim ds = 1 = 1- lim P {X{1) - X{1 - S) = 2) 

5-^l 
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which corresponds to the first condition (|2.29p . For the second condition, let 
n S N*. We have to prove that, for any 77 > 0, there exist 6 and No G N such 
that 

P (w'^iS) >e) <V for N > Nq, (2.30) 
where w'^ is the modulus of continuity defined by 

z«^((5)= sup {\Z^{t)-Z''{t,)\A\Z''ih)^Z^{t)\}. 

0<ti<t<i2<l 

t2-ti<S 

Let us observe that w'^{d) takes either the value or the value 2. Hence, for all 
n > 1, we define Q,, = flLoi^fc+i " '^k > Therefore 

P iw';,{S) > 6) = P {w'^{6) = 2) = P I y [{Tf+i - <S}n {Tf+i < 1} 

\k>0 

= 1 - P(riA^iV(i)) < 1 - p (f^AA"(i) n < n}) 

< l-P(17„n{A/'^(l) <n}). (2.31) 
We set P(0„) = pi^s where pi^s is defined by the following probability 

seN; i=±l 



Introducing St = max sup (pi{t), we obtain the lower-bound 



*=±ite[o,T] 



n ))F(i7„_in{r,f = .,z^(T„^) = z}) 

/ \ L— J 

>(i-^St] " P(o„^in{r,f = s,z^(T,f) = z}). 



Consequently 



P(f^n) > ( 1 - ^^t") ^ F(f^„-l) 



and, by induction, the following lower-bound holds 



T 



liminf P(17„) > lim ( 1 - — 5t ) ^p-"'55T_ (2.32) 

Let us now use the bounds (|2.32p and (|2.3ip in order to prove (|2.30p . Since 
A^^(l) converges in distribution to (see Step 1) as iV 00, there exist 

Nq and n such that 

F{Af^{l) > 72) < rj/2, for aU N > Nq. 

Choose S > small enough such that 1 — e^"'''^^ < 77/2 permits to deduce 
P(f2„) > 1 — 77/2 from (|2.32p . In particular, there exist 5 and Nq such that 

V7V > iVo, 

P(f^„n{A/'^(l) <n}) > l-?7 
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and therefore (|2.3ip leads to (|2.30p . Both conditions needed in Theorem 13.3 
[T] are satisfied, we conlude that the distribution of the process Af^ is tight. 
Step 3. The two first steps (convergence of marginals and tightness) imply 
that converges in distribution towards J\f a.s N becomes large. Let us 
now deduce the convergence of Z towards X. It suffices to use the function 
(j) : N ^ — cos{nAf) which is continuous with respect to the Skorohod topology: 

Z^^(f>{Af^) and X = 0(AA). 

The convergence of is then immediate. □ 



Remark 2.10. The statement of Provosition \2M can he improved: not only the 
process Z^ converges to X , the couple {Z^ ,Af^) (Markov chain and associated 
counting process ) also converges in distribution to (X, Af) . 

Since both the periodic stationary probability measure and the distribution 
of the periodic discrete-time Markov chain converges, we can obtain the large 
time asymptotic behaviour of the statistics of transitions for the time-continuous 
Markov chain via the discretization procedure. The main result announced in 
the introduction is an adaptation of the following statement which is a conse- 
quence of Theorem 12.31 

Let us recall that i/j'^iVif^) (resp. ip{ri,t)) is the moment generating function of 
the transitions (from state —1 to 1) for the discrete-time Markov chain (resp. 
the continuous-time one). 

Theorem 2.11. 1. For all t > 0, the statistics of transitions converge in 
distribution as iV —>■ oo / the moment generating functions satisfy 

V-^l^^L^J) i'iv.t). (2.33) 

2. The eigenvalues of the matrix Ai = liniAr^ooA^^ defined in (|2.12p and 
(j2.13p satisfy Ai > 1 > A2 > and the largest one Ai is given by 

logAi ==log(77) /" v'_(s)M_(s)d5 = log(77)E^[A/-T]. (2.34) 
Jo 

Here 11 denotes the PSPM defined by (jl.2p . The long time asymptotic behaviour 
of the generating function is given by 

\og^{t,r]) log?7 f'^ 

hm = -— / ip^{s)fi^{s)ds. 

t^oo t T Jq 

Proof. By Proposition 12.91 the process J\f^ converges in distribution to J\f. 
The convergence of the generating function is an immediate consequence. In 
particular, 

^^(7?,0) VX^,0) and ^^(7?,7V) ^(7?,T). (2.35) 

We deduce that the four coefficients of the monodromy matrix converge to 
]E[rA<^'l{A>=a}|^o = 6] with (a,6)e{-l,l}2. 



14 



Using the equation ^(77, T) = A4^{ri,0), the Umit matrix Ai is in fact the 
monodromy matrix of the ode (|2.2p . Both eigenvalues of converge to the 
eigenvalues of M. Hence, according to equation (|2.14p 



N-l 



log Af = log(?7) 



fe=0 



which converges to 



log Ai = log(77) / (y9_(s)^_(s)ds 







Since Ai > 1 > A2, the inequality Ai > 1 > A2 holds. By similar arguments as 
those presented in the proof of Corollarv l2.41 we obtain Ai > 1 > A2 and we can 
write the moment generating function in the Floquet basis as follows: ^'(77, t) — 
riui{t) + r2U2{t) with ri ^ 0. Hence '^{r],kT) = AjriMi(O) + A^r2-U2(0). In 
order to obtain the large time asymptotics, let us note that, for any t > 0, there 
exists k £ N such that kT < t < (fc + 1) T and consequently 



Both bounds tend to logAi/T. 

Finally let us note that logA]^ can easily be expressed as the mean number 
of transitions during one period and starting with the PSPM. This identity 
remains true in the large N limit. Indeed it suffices to use the convergence of the 
generating functions to deduce that the family of random variables {N'J^)n>o 
is uniformly integrable. Therefore applying Vallee-Poussin's theorem (see for 
instance Theorem T22 in [TU]) to the function t ^ rf with 77 > 1. We thus 
obtain the convergence of the average number of transitions starting from any 
initial distribution and in particular starting from the PSPM. So we deduce: 



3 Two examples in the stochastic resonance frame- 
work 

We seek to describe the phenomenon of stochastic resonance. The continuous- 
time Markov chain Xt oscillates between two values ±1 according to a T-periodic 
infinitesimal generator Qt. Then by varying the period, we observe that the 
behaviour of the chain changes and adopts more or less periodic paths. The 
aim in each example is to find the optimal period such that the behaviour of the 
paths looks like the most periodic as possible. That's why we shall introduce 
a criterion which measures the periodicity of any random path. We propose to 
use a criterion associated with the largest Floquet exponent of the generating 
function. The interesting tunings correspond to situations where this exponent 
is close to the value log(77). Such a criterion was already presented in |13) . 



log*(77, fcT) log*(77,t) log*(77, (fc + l)r) 
(fc + l)T - t - kf 




□ 
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3.1 An infinitesimal generator constant on each half pe- 
riod 

In this first example, we consider T-periodic rates given by 

<y9„(i) = (^ol{0<t<T/2} + </'ll{T/2<t<T} ^ + (f+it). (3.1) 

where ipo — pe~'^ et (pi = qe~'^ , v < V. This Markov model is often used in 
the stochastic resonance framework (see for instance Here we can compute 

explicitly the invariant measure (see also [TT] Proposition 4.1.2 p. 34) 

Lemma 3.1. The periodic stationary probability measure PSPM is given by: 
and fi^{t) + fi+{t) = 1, pi±{t + T/2) = pL^{t). 

Proof. Using the description of the PSPM in Proposition 11.21 we obtain 

Li-(t) = u_(0)e-('^«+'^i)* + fl - e-'^^^'+f^A 

= L-(0) ^~{vo+vi)t ^ ^ 0<i<r/2. (3.3) 

Furthermore, by symmetry arguments, the dynamics of the periodic invariant 
measure satisfies: ii±{t + r/2) = iJ,zp{t) for all t > 0. We deduce in particular 
that /^-(r/2) = ^+(0) = 1 - Ai-(O). Thus 

' {^0 + + e-(vo+vi)T/2) 

The equation p.3p then permits to conclude. □ 

An immediate consequence of Theorem 12.111 and Lemma 13.11 leads to the 
explicit computation of the largest Floquet exponent (the details of the proof 
are left to the reader). 



Proposition 3.2. The largest Floquet exponent of the ode (|2.2p with the rates 
p.ip . which corresponds to the asymptotic behavior of the generating function 
of the statistics of transitions Mt, is given by log(?7)Ep[A/'T] where 

^,Wt] f^)\,nh(i^o + ^,)T/i). (3.4) 

We are interested in the phenomenon of stochastic resonance associated to 
continuous-time process (X(, t > 0). This process essentially depends on two 
parameters: a parameter e describing the intensity of the transition rates be- 
tween both states {—1,-1-1} (some small e corresponds to a frozen situation: the 
Markov chain remains in the same state for a long while) and a second parameter 
T, the period of the process dynamics. By considering the normalized process 
Yt = XtTj especially its paths on a fixed interval [0, S'], we observe the following 
phenomenon (for fixed e): if T is small then there are very few transitions of 
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Y: the process tends to remain in its original state. If T is large, Y behaves in 
a chaotic way: lots of transitions are observed. For some intermediate values 
of T, the random paths of Y are close to deterministic periodic functions (one 
transition in each direction per period). Let us note that this phenomenon can 
also be observed by freezing the period length T and varying the intensity e of 
the rates. 

The aim is therefore to point out the best relationship (tuning) between e 
and T which makes the process Y the most periodic as possible. If the process 
is close to a periodic function then the number of transition from state — 1 to 
sate +1 is close to 1 per period, which leads to find the tuning corresponding 
to the Floquet exponent equal to log 77. By Proposition [321 it is then sufficient 
to find the best relation between e and T such that 

E^[AAt] = 1. (3.5) 

In Figure [XTl we set e = 0.1, V ^ 2, v ^ 1, and let T vary. We compute nu- 



average number of Iransilions versus T 
1 ,005 n 




0.980 - 



975 H ' 1 ' 1 ' 1 ' 1 ' 1 ' 1 ' 1 ' 1 ' 

2.0e+005 4.0e+005 6.0e+0OS e.Oe+005 I.Oe+OOB 1.3e+O06 1,46+006 1,6e+006 1,86+006 3,06+006 



Figure 1: Average number of transitions 

merically the average number of transitions per period. We can clearly observe 
that there is one and only one period corresponding to the condition 



Proposition 3.3. Let T^p^ be the period which provides an average number of 
transitions per period equal to 1 . The following asymptotic behaviour holds, as 
e tends to 0, 



^opt-^e''/^ (3.6) 



V -V 

Proof. The condition p.5p combined with Proposition 13.21 leads to the equation 



tanh((^o + <^i)T/4) = 1. 



The aim is to solve it and let e tend to 0. The left member in the previous 
equation is an increasing function of T. We introduce the change of variable 
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If^ = {(po + cpi)T/A. We first prove that [/^ increases as e decreases. If^ satisfies 
K{U\e) = 1 with 



K{U\ e) = + ^^^^^ tanh(C/'). 

Both functions e i— J- iir(-,e) and x i— > -ftr(x, •) decrease for e small and a; large 
enough. It follows that increases when e decreases and tends to 0. Let us 
assume ^ Uq < oo in the limit e 0. Then K{U^,e) tanh(C/o) which 
contradicts the identity K{U'^,e) ~ 1. We deduce that If^ oo when e ^ 0. 
Now let us set t = e~^l^ and fi = vjV < 1. With these new parameters K can 
be written like 



1 = K(U, t) = ^ + ^ tanh(C/n. 

Let us define tanh(C/') ~.1~W then C/' = 5 log (^^), we obtain that W 
tends to when i — > and the previous equation becomes: 

Thus, when t — > 0, we have 

K(W, - 1 - (^^^^ (log ^ + 0(108 W)) 

+ (l_lPtl-'9 + o(tl-'9))(l_M/)_l 



9 

-2pqt^+^ 
{pt + qtPy 



\ogW -W + o{t^-^ log W)=0 



liW = rot" \og{t)R{t) with a = 1 - /? and j'o = - ^ = - y(1 - /?), we obtain 
the limit R{t) 1 when t ^ and therefore 



f/^--2log^f (l-/3)t-^(-logt)j^-^logi 2^ 2^ . 

We recall [/' = (ipQ + ipi)T/A which leads to the result set. □ 

In |11| , several quality measures have been proposed to point out the optimal 
tuning of Y: the spectral power amplification (SPA), the SPA to noise intensity 
ratio (SPN). the energy (En), the energy to noise intensity ratio (ENR), the out- 
of-phase measure which describes the time spent in the most attractive state, 
the entropy or relative entropy. In his PhD report, I. Pavlyukevich computes for 
each measure the optimal relation between e and T^^^, the length of the period, 
in the small e limit, we adopt a similar procedure in Proposition [231 So we can 
now gather these quality measures into three families: 

• for the first family, the optimal tuning satisfies T^gs = "^(T^pj) where T^p^ 
is given by p.6p . The associated Markov chain has an average number of 
transitions from —1 to +1 strictly smaller than 1. This family contains in 
particular the SPN. 
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• The second family concerns T^^^ = o{T^^^). The Markov chain has then 
more than one transition per period on average. This family contains most 
of the measures: SPA, En, Out-of-phase, the entropy and relative entropy. 

• Finally in the third family T^pt and T^os comparable, this is namely 
the case for ENR. 



3.2 Infinitesimal generator with constant trace 

Let us finally present a second example of periodic forcing in the stochastic 
resonance framework. This model was introduced by Eckmann and Thomas 
[3]. The aim in this paragraph is to find the optimal tuning between the noise 
intensity in the system and the period length in order to reach an average 
number of transitions during one period close to 1. This approach is different 
from the study prensented in [J. 

The model consists in a continuous-time Markov chain with periodic forcing: 
the transition rates are given by 

ip-{t) ^ e {a + cos wt) and = e(a — cos wt), a > 1. (3.7) 

The period satisfies T = (27r)/aj. In this particular case, the trace of the in- 
finitesimal generator, defined by (|0.1|) . is a constant function. It is then quite 
simple to compute explicitely the periodic stationary probability measure and 
the Floquet multipliers associated with the moment generating function of the 
statistics of transitions. 

Lemma 3.4. The periodic stationary probability measure of the periodic forced 
Markov chain is given by 

1 e 
Proof. Using Proposition [TT^l we obtain 



fj-~(t) = - — ^ „ „ ^(2aecosajt + ujsmujt). (3.8) 

O /I ^ZfzZ _[_ /, ,z 



At_(t) =/i_(0)e-2<^'^*4- / {ea-ecosujs)e-^"'^'-''>ds. 
Jo 

Hence 

2-6^2""* 2e'^ ae-^""-* ~2e'^ a COS ujt- ecu sin Lut 
fi-it) = + + .22^2 • 

Setting IJ,-(T) = /i-(0), we obtain /J.-(0) = - — „ „ 5- and consequently 

2 4e^a^ 4- 

the announced statement. □ 

An application of Theorem 12.111 permits to describe the large time asymp- 
totics for the moment generating function of the transitions from state —1 to 
state -t-1. It suffices to compute explicitely / = (p-(t)fX-{t)dt. The result is 
described in the following statement while the proof is left to the reader. 

Proposition 3.5. The largest Floquet exponent associated with the statistics of 
transitions Afx (the moment generating function) is equal to log{ri)lE^[N'T] with 

^^[■^-J- — -WT^' (3-9) 

and fi given by 
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Let us now discuss the suitable choice of the period such that Ep[A/'t] = 1- 
We then need to solve 

7rea(4e2a2 + cj^) - 27re^a = ^(46^0^ + u^). (3.10) 

It is obvious that uj is of the order e, we set cj = /ie and look for the best choice 
of the parameter /i. Considering (j3.10p . the optimal value /i is in fact a real 
root of the following polynomial function 

P(/.i) := /.i^ - ■Ka^? + 4a^/i + 27ra(l - 2a^) 

It is straightforward to prove that this polynomial function has a single positive 
root since it is increasing and verifies P(0) < 0. Using the Cardan formula, 
we can obtain an explicit expression of /^optimal which depends of course on the 
coefficient a, this dependence is asymptotically linear as a becomes large. 
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